Vehicle control by a remote operator

ABSTRACT

A computer implemented vehicle control method may include obtaining a sensed input from an operator remote from a vehicle, recognizing and associating the sensed input with a vehicle action and outputting control signals to the vehicle to cause the vehicle to carry out the vehicle action.

CROSS-REFERENCE TO RELATED PATENT APPLICATIONS

The present application claims priority under 35 USC § 119 fromco-pending U.S. Provisional Patent Application Ser. No. 62/962,752 filedon Jan. 17, 2020 by Whitney et al. and entitled VEHICLE CONTROL BY AREMOTE OPERATOR, the full disclosure of which is hereby incorporated byreference.

BACKGROUND

Vehicles may perform various vehicle actions in response to inputs orcommands from an operator. For example, the direction in which a vehicletravels and/or speed of travel may be controlled by an operator sittingin the vehicle and manipulating a steering wheel, joystick, acceleratorpedal, brake pedal and the like. Various attachments or extensions ofthe vehicle may also be controlled from the operator sitting orotherwise boarded upon the vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram schematically illustrating portions of anexample vehicle control system.

FIG. 2 is a flow diagram of an example vehicle control method.

FIG. 3 is a block diagram schematically illustrating portions of anexample vehicle control system.

FIG. 4 is a block diagram schematically illustrating portions of anexample vehicle control system.

FIG. 5 is a front perspective view of an example vehicle control system.

FIG. 6 is a rear perspective view of the example vehicle control system.

FIG. 7 is a right-side view of the example vehicle control system.

FIG. 8 is a left side view of the example vehicle control system.

FIG. 9 is a front view of the example vehicle control system.

FIG. 10 right side rear view of the example vehicle control system.

FIG. 11 is a bottom perspective view of the example vehicle controlsystem taken along line 11-11 of FIG. 2.

Throughout the drawings, identical reference numbers designate similar,but not necessarily identical, elements. The figures are not necessarilyto scale, and the size of some parts may be exaggerated to more clearlyillustrate the example shown. Moreover, the drawings provide examplesand/or implementations consistent with the description; however, thedescription is not limited to the examples and/or implementationsprovided in the drawings.

DETAILED DESCRIPTION OF EXAMPLES

Disclosed are vehicle control systems, methods and mediums thatfacilitate control of a vehicle by an operator remote from the vehicle.As a result, vehicle actions that might otherwise demand multiplepersons, one person remote from the vehicle on the ground and anotherperson boarded upon the vehicle and controlling the vehicle may becarried out by a single operator. As a result, vehicle actions thatmight otherwise demand that an operator boarding the vehicle repeatedlyleave the vehicle to get a new perspective then re-boarding the vehiclemay be performed with fewer or no re-boards.

For purposes of disclosure, the term “remote”, when referring to anoperator, means that the operator is locally located with respect to thedirect control devices of the vehicle such as a steering wheel,joystick, acceleration or brake pedals, gearshift levers, push buttons,switches, other levers and the like. In some implementations, a remoteoperator may be an operator that is standing on a part of the vehicle oran attachment of the vehicle, but wherein the operator cannot access thesteering wheel or other input controls of the vehicle. For example, anoperator may be located on a platform, bucket, or other attachment ofthe vehicle, but cannot readily ask the steering will other inputcontrols of the vehicle. In some implementations, a remote operator maybe an operator that is not boarded upon the vehicle. For example, aremote operator may be an operator standing on the ground in front of,behind of or to a side of the vehicle. In some implementations, a remoteoperator may be an operator that is standing or otherwise carried by animplement being pushed or pulled by the vehicle. In each case, theoperator is remote from the direct control structures (steering wheel,joystick, push buttons, switches, levers, and the like) of the vehicle.

In some implementations, a remote operator may be an operator thatcannot readily reach, contact or physically access the direct inputinterfaces for a particular vehicle action, wherein disclosed vehiclecontrol systems, methods and mediums facilitate initiating, stopping oradjusting such vehicle actions by an operator through the use of director indirect gestures by the operator that are sensed by a sensor of thevehicle. Direct gestures comprise movement or positioning of theoperator's anatomy such as movement or positioning of the operator'shands, arms, or legs. Indirect gestures of an operator may comprisemanual movement or positioning of an input device by an operator,wherein the movement or positioning of the input device is sensed bysensor of the vehicle. In such circumstances, an operator who ispositioned so as to not be able to physically contact and move a directinput device (steering wheel, joystick, push buttons, switches, leversand the like) for a desired vehicle action may still provide input tothe vehicle initiating, stopping or adjusting such a vehicle action.

FIG. 1 is a block diagram schematically illustrating an example vehiclecontrol system 20. Vehicle control system 20 comprises vehicle 24,sensor 28, processor 32 and a non-transitory computer-readable medium40. Vehicle 24 comprises a self-propelled vehicle. Examples of vehicle24 include, but not limited to, trucks, cars, tractors, harvesters,riding lawnmowers, snow throwers, four wheelers, all-terrain vehicles,and the like.

Sensor 28 comprises at least one sensor carried by vehicle 24 that issupported by vehicle 24 so as to be able to sense direct or indirectgestures initiated by an operator 42. The direct gestures provided byoperator 42 may be provided by the operator's anatomy 44, such as amovement or positioning of the operator's hands, fingers, legs, torso,or the like. The movement, positioning/orientation of the operator'sanatomy may serve as input 46 which is sensed by sensor 28. Indirectgestures initiated by operator 42 may involve the movement and/orpositioning of an input device 48 which serves as input 46. The inputdevice 48 may comprise a flag, a baton, a smart phone or other handheldor portable physical structure that may be manually manipulated by theoperator 42 and that is recognizable by sensor 28.

Sensor 28 may have varying fields of view or sensing ranges. In someimplementations, particular regions about vehicle 24 that are within theparticular field of view of at least one of sensors 28 may be designatedfor providing remote input to vehicle 24. In other implementations, theat least one sensor 28 may have a field-of-view or multiple fields ofview that encompass an entire area about vehicle 24 such that a remoteoperator may provide remote input at any of various possible locationsabout vehicle 24.

In some implementations, sensor 28 comprises at least one camerasupported by vehicle 24. In other implementations, sensor 28 maycomprise other forms of non-contact or wireless sensors such as lidar,radar, ultrasonic sensors, and the like. In some implementations,different types of sensors may be provided at different locations aboutthe vehicle.

Processor 32 and medium 40 form a controller for vehicle 24. Althoughprocessor 32 and medium 40 are illustrated as being part of or carriedby vehicle 24, in some implementations, processor 32 and medium 40 maybe located remote from vehicle 24, not being carried by vehicle 24. Insuch implementations, the controller formed by processor 32 and medium40 may communicate with a local controller on vehicle 24 in a wirelessmanner. Processor 32 carries out instructions provided in medium 40.Medium 40 may contain additional instructions (not illustrated) forcontrolling other operations of vehicle 24.

Medium 40 may be in the form of software or coding on a flash drive,memory disk, or the like and/or hardware in the form of logic elementson a circuit board. The instructions contained in medium 40, that directprocessor 32, comprise remote operator input sensing instructions 56,input recognition instructions 58 and input response controlinstructions 64. Remote operator input sensing instructions 56 compriseinstructions configured to obtain sensed input from the remote operator42. Such instruction direct processor 32 to pull or otherwise acquiresignals from sensor 28 indicating the positioning and/or movement ofoperator anatomy 44 and/or input device 48. Instructions 56 may furtherdirect processor 32 to determine such positioning or movement from thesignals provided by sensor 28.

In one implementation, system 20 is selectively actuatable betweendifferent modes. In a first mode, sensor 28 may sense the positioningand/or movement of operator anatomy 44 and use such positioning and/ormovement to control actions of vehicle 24. In a second mode, sensor 28may sense the positioning and/or movement of input device 48 and usesuch positioning and/or movement to control actions of vehicle 24. Insome implementations, the acquisition of signals from sensor 28 forfacilitating remote control of vehicle 24 and/or the generation ofcontrol signals for one or more vehicle actions based upon a sensedgesture of operator 42 may be continuous or may be initiated in responseto an input provided by the operator 42 through a direct input control,such as while the operator is boarded upon vehicle 24. In someimplementations, the acquisition of signals from sensor 28 forfacilitating remote control of vehicle 24 and/or the generation ofcontrol signals for one or more vehicle actions based upon a sensedgesture of operator 42 may be triggered or initiated in response tosignals indicating that the operator is no longer boarded upon vehicle24. For example, in some implementations, sensor 28 and/or theremote-control process of system 20 may be in a dormant mode and may bewoken in response to signals from a sensor indicating that the operatorhas left vehicle 24. In one implementation, one or more sensors may belocated below the operator's chair or seat in vehicle 24, wherein thesensor 28 and/or the remote control process provided by system 20 may beawoken in response to such sensors indicating that the operator is nolonger seated.

Input recognition instructions 58 comprise instructions configured todirect processor 32 to recognize and associate sensed input with aparticular requested vehicle action. For example, an operator 42pointing his or her hand in a downward direction may be interpreted as acommand to decrease the forward or rearward velocity of the vehicle,whereas the operator 42 pointing his hand in an upward direction may beinterpreted as a command to increase the forward or rearward velocity.An operator 42 pointing input device 48 in a downward direction may beinterpreted as a command to decrease the forward or rearward velocity ofthe vehicle, whereas the operator 42 pointing the input device 48 in anupward direction may be interpreted as command to increase the forwardor rearward velocity. The operator 42 pointing his hand in a leftwarddirection or pointing the input device 48 in the leftward direction maybe interpreted as a command to turn the vehicle in a leftward direction,wherein the duration which the hand is pointed in the left directionindicates the extent or angle of the turn. An operator pointing his orher hand or input device directly at vehicle 24 may be interpreted as acommand to back up the vehicle.

In one implementation, input recognition instructions 58 may directprocessor 32 to discern between the operator's left-hand and right-hand,wherein different gestures provided by the left-hander the right hand(or an input device carried by the left-hand or right-hand), may beinterpreted as different commands. For example, gestured by theleft-hand may be interpreted as providing commands for the speed of thevehicle whereas gestures provided by the right hand may be interpretedas providing commands for movement of an attachment or implement of thevehicle.

In one implementation, medium 40 may additionally include a database orlookup table associating different sensed inputs (different sensedgestures) with different vehicle commands or actions. In someimplementations, the database may be local, carried by vehicle 24. Inother implementations, the database may be remote from vehicle 24. Insome implementations, the database may be a generic database provided bya remote server, wherein the database is accessible to multipledifferent vehicles 24 and different systems 20 being operated bydifferent operators 42. In some implementations, the database may bespecific to the particular operator 42. In some implementations, thedatabase may be part of a neural network that is been trained usingimages, videos or other sets of sensed data, or the neural networkrecognizes different gestures and associates such gestures withdifferent vehicle action commands or requests.

In some implementations, the input recognition instructions 58 may havedifferent databases of associated commands and gestures for differentindividual sensors 28 supported at different positions by vehicle 24. Agesture received from a first sensor at a first location may correspondto a first vehicle action while the same gesture received from a secondsensor at a second different location may correspond to a seconddifferent vehicle action. In some implementations, different sensorssupported at different locations or positions on the vehicle may bededicated or assigned to different vehicle actions. For example, a firstsensor or group of sensors at a first location on a vehicle may bededicated to receiving direct or indirect gestures for controlling afirst type of vehicle action while a second sensor or a second group ofsensors at a second location on the vehicle may be dedicated toreceiving direct or indirect gestures for controlling a second differenttype of vehicle action. By way of a more specific example, a firstsensor supported at a front end of a vehicle may be dedicated toreceiving direct or indirect gestures for controlling the positioning ofan attachment extending from the front of the vehicle, whereas a secondsensor supported at a rear end of the vehicle may be dedicated toreceiving direct or indirect gestures for controlling the positioning ofan attachment extending from the rear of the vehicle. In someimplementations, for an operator to provide remote input for aparticular vehicle action, the operator must position himself or herselfat a pre-determined or designated remote location relative to thevehicle such that his or her direct or indirect gestures are captured bythe appropriate sensor that is designated for the desired vehicleaction.

With such example implementations, an operator may be prevented frominadvertently providing an incorrect gesture for an incorrect command.For example, in one implementation, gestures associated with forwardmovement of the harvester may be only received from sensors positionedalong or facing a side or rear of the vehicle 24. Sensors facing a frontto vehicle 24 may be dedicated to other vehicle actions, but not forwardmovement of vehicle 24. In such implementations, the operator may berequired to be along a side or rear of the vehicle, rather than in frontof the vehicle when instructing forward movement of the vehicle.

By way of another example, one or more sensors having a field-of-viewencompassing a power take off of the vehicle may be blocked or not beassociated with receiving gestures corresponding to commands to turn onthe power take off. In other words, only sensed gestures from sensors 28having a field-of-view sufficiently distant from the power take off maybe used to turn on the power take off. In such implementations, suchassignment of sensors to particular vehicle actions may prevent anoperator from becoming accidentally entangled in the power take off.

In some implementations, the association of different gestures withdifferent requests or commands for particular vehicle actions may beadditionally based upon other sensed parameters. For example, whenvehicle 24 is carrying out a first operation or is in a first date (assensed by sensors or otherwise determined by processor 32), recognitioninstructions 58 may direct processor 32 to consult a first table ordatabase of gesture-vehicle action associations. When vehicle 24 iscarrying out a second different operation or is in a second differentstate, recognition instructions 58 may direct processor 32 to consult asecond table or database containing different gesture-vehicle actionassociations. By way of a specific example, instructions 58 may directprocessor 32 to consult different databases containing differentgesture-vehicle action associations depending upon the type orcharacteristics of attachment connected to vehicle 24 or depending uponthe type or characteristic of the particular implement currently beingpushed or pulled by vehicle 24. The type or characteristics of theattachment or implement may be input by the operator or may be sensed.

Input response control instructions 64 comprise instructions configuredto output control signals to various actuators or the like of vehicle 24to cause vehicle 24 to carry out the particular vehicle actioncorresponding to the sensed input as determined by instructions 58.Examples of various vehicle actions that may be associated withparticular gestures (direct or indirect) from operator 42 in which maybe carried out in response thereto include, but are not limited tovehicle actions consisting of: forward velocity, backward velocity,left/right direction, braking, lights (nightlights, running lights,spotlights), signal, sound (horn, loudspeaker), warning (flashinglights, hazard lights), implement specific actions (left sprayer on/off,right sprayer on/off, left implement wing raising and lowering, rightimplement wing raising and lowering, power take-up, moving a dischargespout, changing operational speed of the auger of a discharge spout,turning on/off of a power take off, adjusting a speed of the powertakeoff, raising/lowering an attachment to the vehicle (such as abucket, fork or the like), adjusting the supply of hydraulic fluid orhydraulic power to implement or attachment, raising/lowering a threepoint hitch in the like.

FIG. 2 is a flow diagram of an example vehicle control method 120.Although method 120 is described in the context of being carried out bysystem 20, it should be appreciated method 120 may likewise be carriedout with any of the following described systems or with similar systems.As indicated by block 124, processor 32 may obtain a sensed input froman operator 42 remote from vehicle 24. The sensed input may be acquiredfrom at least one sensor 28 carried by vehicle 24.

As indicated by block 128, processor 32 may recognize and associate thesensed input with a particular requested or commanded vehicle action. Asdescribed above, such association may be through the consultation of alocal or remote database or lookup table associating different sensedinputs/gestures with different vehicle actions. In some implementations,the determination of the particular requested vehicle actioncorresponding to the sense input or gesture may additionally be basedupon from which particular sensor 28 the sensed gesture was receivedand/or the particular state of vehicle 24, including the state orcharacteristic of any implement or attachment associated with vehicle24. In some implementations, the operator 42 or other manager mayprovide system 20 with selections identifying which particular vehicleactions may be requested through the use of remote sensed gestures. Forexample, a database may include a multitude of available vehicle actionsthat may be controlled through the use of remote gestures, but where theoperator or another person may authorize only a portion or a selectedgroup of such available vehicle actions for control through remotegestures.

As indicated by block 132, processor 32 may output control signals tothe vehicle 24 to cause a vehicle 24 to carry out the particular vehicleaction currently associated with the sensed input/gesture from theremote operator 42. As described above, examples of such vehicle actionsinclude, but are not limited to, forward velocity, backward velocity,left/right direction, braking, lights (nightlights, running lights,spotlights), signal, sound (horn, loudspeaker), warning (flashinglights, hazard lights), implement specific actions (left sprayer on/off,right sprayer on/off, left implement wing raising and lowering, rightimplement wing raising and lowering, power take-up, moving a dischargespout, changing operational speed of the auger of a discharge spout,turning on/off of a power take off, adjusting a speed of the powertakeoff, raising/lowering an attachment to the vehicle (such as abucket, fork or the like), adjusting the supply of hydraulic fluid orhydraulic power to an implement or attachment, raising/lowering of athree point hitch and the like.

FIG. 3 is a block diagram schematically illustrated portions of anexample vehicle control system 220. Vehicle control system 220 issimilar to vehicle control system 20 described above except that vehiclecontrol system 220 comprises vehicle 224 and input device 248 in placeof vehicle 24 and input device 48. Vehicle 224 the additionallyillustrated as being coupled to an attachment/implement 225.

Vehicle 224 is itself similar to vehicle 24 except that vehicle 224 isillustrated as specifically comprising lights 300, steering unit 302,propulsion unit 304, power take off (PTO) unit 306, hydraulic power unit308, brakes 310 and auxiliary units 312. Vehicle 224 additionallycomprises input-action store 314, authorization store 316, microphone318 and sensor 320. Medium 40 additionally comprises operatoridentification and authorization instructions 52, input deviceidentification and authorization instructions 54 and operator positionidentification instructions 60. The remaining components of vehicle 224and system 220 which correspond to components of system 20 are numberedsimilarly.

Lights 300 comprise lights supported by vehicle 224 for providingillumination about vehicle 224 or for providing alerts or notificationsfor vehicle 224. Steering unit 302 comprises electrical and/or hydrauliccomponents and associated controllers that effectuate turning of thewheels, tracks, or the like to steer forward or rearward travel ofvehicle 224. Propulsion unit 304 comprises an internal combustionengine, electric motor, transmission, and associated controllers forcontrolling the forward and rearward propulsion of vehicle 224. PTO unit306 comprises an electrical, hydraulic, or mechanical drive andassociate controllers for rotating the power take off (such as aprojecting spline) for supplying torque to a fitting associated with anattachment or implement. Hydraulic power unit 308 comprises hydraulicpumps, valves, and associated controllers for supplying pressurizedhydraulic fluid to portions of vehicle 224 or to attachments/implementspowered by such pressurized hydraulic fluid from vehicle 224. Brakes 310comprise devices for braking, slowing down the propulsion of vehicle224. Auxiliary units 312 comprise movable or actuator components ofvehicle 224. For example, auxiliary units 312 may comprise dischargespouts of a harvester, wherein the positioning of the discharge spoutand/or the rotation of an auger of the discharge spout are adjustable.

Attachment/implement 225 comprises an attachment carried by vehicle 224and/or an implement being pushed or pulled by vehicle 224. An attachmentmay be in the form of a bucket, blade, harvester head or the like.Examples of an implement may include any of a variety of implement suchas wagons, carts, plows, discs, choppers, balers, sprayers, and thelike. As discussed above, vehicle actions may involve repositioning suchattachments are implements or adjusting the supply of power to suchattachments or implements. The association of particular gestures toparticular inputs/commands may vary depending upon what particularattachment implement is coupled to vehicle 224 and/or the current stateof the particular attachment or implement coupled to vehicle 224. Thesame director indirect gesture may be associated with different commandsdepending upon the particular attachment or implement coupled to vehicle224 and/or the current state of the particular attachment or implementcoupled to vehicle 224.

Input-action store 314 comprises one or more databases or lookup tableslinking various sensed gestures (direct or indirect) to associatedrequests or commands for vehicle actions.

Authorization store 316 comprises one or more databases or lookup tablesidentifying preauthorized operators and/or preauthorized input devices248 for providing gestures for inputting requests or commands forvehicle actions. For example, authorization store 316 may comprisephotographs of authorized operators 42, wherein authorization of anoperator may be determined by comparing captured images of a candidateoperator 42 and the photographs contained in the store 316.Authorization store 316 may comprise a pre-assigned set of passwords,wherein authorization for an operator 42 or an input device 248 may bedetermined by comparing a received password input through microphone 318to the store 316. Authorization store 316 may comprise barcode values orother signatures for authorize input devices 248. Input-action store 314and authorization store 316 may be contained on medium 540 carried byvehicle 524 or may be stored in a remote memory or server, whereinvehicle 524 accesses stores 314, 316 through a wireless communicationconnection with the remote memory or server.

Operator identification and authorization instructions 52 compriseinstructions for directing processor 32 to identify and authorize acandidate operator 42 for providing direct gestures for providing remotecontrol commands for vehicle 224. Instructions 52 may direct sensor 28or an alternative sensor, such as sensor 320 (in the form of a camera orother sensor) to capture images of operator 42 and then compare thereceived information or data to information found in authorization store316. Based on such comparison, the operator 42 may be authorized forproviding direct gestures for use in remotely controlling vehicle 224.

Input device identification and authorization instructions 54 compriseinstructions for directing processor 32 to identify and authorize acandidate input device 248 doe providing direct gestures for providingremote control commands for vehicle 224. Instructions 54 may directsensor 28 or an alternative sensor, such as sensor 320 (in the form of acamera or other sensor) to capture images a barcode or other indicia ofinput device 248, or receive an identification/authorization signal frominput device 248, and then compare the received information or data toinformation found in authorization store 316. Based on such comparison,the input device 248 may be authorized for providing indirect gesturesfor use in remotely controlling vehicle 224.

Operator position identification instructions 60 comprise instructionsthat direct processor 32 to identify the positioning of the remoteoperator 42 relative to vehicle 224. Based upon the determined relativepositioning, such instructions may further direct processor 32 to eitheroutputting notification to the operator 42 recommending that theoperator move relative to the vehicle or automatically interrupt therequested vehicle action corresponding to the sensed operatorinput/gesture. In such a fashion, instructions 60 may prevent vehicleactions from being carried out when the operator may be too close or outof position with respect to vehicle 224 for the vehicle action beingrequested.

Input device 248 comprises a handheld device to be manually manipulated,moved, or positioned by operator 42. Input device 248 comprises a firstface 330 having an input identifier 332. Input identifier 332 isrecognizable by sensor 28 and processor 32 following input recognitioninstructions 58. In some implementations, input identifier 332 maycomprise flashing lights, particular patterns or shades of the color orother characteristics readily perceptible by sensor 28 to facilitate thesensing of the positioning and/or movement of input device 248.

Input device 248 additionally comprises a second opposite face 334having a display 336. In one implementation, signals from sensor 28 orsensor 320 may be transmitted to input device 248, wherein a depictionof the region surrounding vehicle 224, based upon such signals, ispresented on display 336. For example, one of sensor 28, 320 maycomprise a camera carried by vehicle 224. The captured images may betransmitted to input device 248 and presented on display 336. As aresult, the operator 42 providing remote commands to vehicle 224 maymake such gestures and provide such commands based upon not only on hisor her perspective which is remote from vehicle 224 but also based uponthe perspective of the sensor 28 or 320 taken from the perspective ofvehicle 224. Thus, the operator may make a more informed decisionsregarding such remote commands. In one implementation, input device 248may comprise a smart phone that wirelessly communicates with thecontroller provided by processor 32 and medium 40, wherein thepositioning or movement of the smart phone serves as a remote gesturefor providing remote commands to vehicle 224.

FIG. 4 is a schematic view illustrated portions of an example vehiclecontrol system 420. As shown by FIG. 4, system 420 allows a driver tocontrol the motion of a tractor, and actions of tools/devices attachedto it whilst being physically removed from the tractor (such as standingin front of it). Sensors are mounted on the tractor so that the tractorcan collect sensor data. The tractor analyzes the data to look forgestures that it has been trained to recognize. The tractor takesappropriate control changes dependent upon the gestures recognized. Thissystem can be used to perform such tasks as instructing a tractor tofollow the driver around a field or positioning a tractor within aclosed space (such as garage).

As shown by FIG. 4, vehicle control system 420 comprises a sensor array428 in the form of a set of sensors that sense and output real-time dataregarding the input provided by operator 42 either through his or heranatomy 44 or through an input device, such as input device 248. Thesensed data is transmitted to neural networks 432 which are trained torecognize a set of control gestures or inputs. Such recognition may bebased upon training library 434 which may comprise a set of videos thatshow control gestures being given.

Vehicle control system 420 may further comprise rules engine 436 whichcomprises a processor and a non-transitory computer-readable medium thatoutputs control instructions for vehicle 224, in the form of a tractor,based upon the gestures or input identified by neural networks 432. Asindicated by block 438, operation of vehicle 224 is adjusted based uponthe control instructions. Such control instructions may involvesteering, velocity and the like as described above. For example, suchcontrol instructions may control the operation of lights 300, steeringunit 302, propulsion unit 304, PTO unit 306, hydraulic power unit 308,brakes 310 and/or auxiliary unit 312.

FIGS. 5-11 illustrate an example vehicle control system 520 for avehicle 524 in the form of a tractor. Vehicle 524 is similar to vehicle224 except that vehicle 524 additionally comprises vehicle state andfeedback system 525. Vehicle state and feedback system 525 provides anoperator, remote from vehicle 524, with visible and/or audible feedbackregarding the state of vehicle 524. Such feedback may include the speedor rate at which the vehicle is traveling, the speed or state of animplement and/or the state of any of lights 300, steering unit 302,propulsion unit 304, PTO unit 308, brakes 310 auxiliary unit 312. Suchfeedback may include a confirmation of receipt or capture of gesturesfrom the operator (either operator anatomy 44 and/or input device 248),a confirmation of recognition of such gestures, an indication that suchcommands are about to be executed, a request for the operator repeatingsuch gestures, and/or an indication that the commands associated withsuch gestures will not be carried out given the current state of vehicle524 or the operator's position relative to vehicle 524 and/or itsimplements.

Vehicle 524 may be used for a variety of purposes in agriculturalconstruction and residential purposes. Vehicle 524 may be used to pushor pull an implement. Vehicle 524 may include attachments, such as abucket, blade, backhoe, or the like for digging, displacing, and/orcarrying various materials such as earthen materials, animal waste andproduce. Vehicle 524 may include forks or other coupling mechanisms forengaging pallets, bins, boxes, or the like, wherein the tractors carryand/or lift the engaged items.

Vehicle 524 comprises chassis 600, ground propulsion members 602,battery 604, and vehicle cab 606. Vehicle 524 further comprises lights300, steering unit 302, propulsion unit 304, PTO unit 306, hydraulicpower unit 308, brakes 310 and auxiliary unit 312. Chassis 600 comprisesa frame supporting the remaining components of vehicle 524. In theexample illustrated, chassis 600 comprises a front cargo bed 608 forstoring and transporting cargo. In the example illustrated, chassis 600is further configured for connection to an attachment/implement 225. Inthe example illustrated, propulsion unit 304 comprises an electric motordriven by electrical power supplied by a battery.

Ground propulsion members 602 comprise members that engage theunderlying terrain and which are driven by propulsion unit 304. In theexample illustrated, ground propulsion members 602 comprise rear wheels610 and front wheels 612. In the example illustrated, rear wheel 610 aredriven by propulsion unit 304 while front wheels 612 are manipulated orturned by steering unit 302. In other implementations, ground propulsionmembers 602 may comprise tracks or other ground engaging members.

Battery 604 comprises a battery unit that is removably received within acorresponding chamber or cavity extending rearwardly from the front ofchassis 600. Battery 604 mates with a corresponding connection interfacefor transferring electrical power from battery 604 to the electricallypowered components of vehicle 524. In other implementations, battery 604may be located at other locations. In other implementations, battery 604may be fixed and non-swappable or not removable. In the exampleillustrated, battery 604 electrically powers propulsion unit 304 whichdrives rear wheel 610. In the example illustrated, battery 604electrically powers hydraulic motors or pumps of hydraulic power unit308, steering unit 302 and brakes 310. Battery 604 additionally powerslights 300, attachment/implement 225, and auxiliary units 312.

Cab 606 comprises a compartment in which an operator may be seated whenoperating vehicle 524. Cab 606 comprises a seat 612, a steering wheel616, a control console 618 and a roof 620. Roof 620 extends over controlseat 612 and control console 618. In some implementation, roof 620 maybe raised and lowered.

Lights 300, steering unit 302, propulsion unit 304, PTO unit 306,hydraulic power unit 308, brakes 310 and auxiliary unit 312 aredescribed above. In the particular example illustrated, PTO unit 306comprises a power take off 623 (shown in FIG. 10). In the exampleillustrated, lights 300 comprise hood lights 624 and roof lights 626.

As with vehicle 224, vehicle 524 include sensors that capture thecontrol gestures made by the operator 42. In the example illustrated,such sensors comprise cameras 530-1 (shown in FIG. 5), 530-2 (shown inFIG. 6) and 530-3 (shown in FIG. 11) (collectively referred to ascameras 530). Cameras 530 capture images of operator control gestures aswell as the surrounding environment and output signals to processor 32.Camera 530-1 extends on a front edge of roof 620 to capture regions infront of vehicle 524. Camera 530-2 extends on a rear edge of roof 620 tocapture images of regions rearward of vehicle 524. Cameras 530-3 extendon underside of roof 620 to capture side regions of vehicle 524. Suchcameras output signals identifying the location of the operator 42. Insome implementations, vehicle 524 may include additional or fewercameras at the same or different locations and alternative forms ofsensors.

Vehicle state and feedback system 525 comprises indicators 570-1, 570-2,570-3, 570-4 (collectively referred to as indicators 570), indicator572, indicator 574, and state/feedback instructions 568. Indicators 570comprise display screens located at the four corners of roof 620.Indicators 570-1 and 570-2 face in a forward direction and are angledtowards their respective opposite sides of vehicle 524. Indicators 570-3and 570-4 face in a rearward direction and are angled towards theirrespective opposite sides of vehicle 524. Indicators 570 presentgraphics and text which may be viewed by the operator 42 at variouspositions about vehicle 524.

Indicator 572 comprises an elongate bar or strip that wraps around afront of the hood 601 and the sides of hood 601 of vehicle 524, whereinthe bar or strip may be selectively illuminated under the control ofprocessor 32. In some implementations, indicator 572 is actuated betweenan illuminated and a non-illuminated state to provide feedback to theoperator 42 who may be remote from vehicle 524, not within cab 606. Insome implementations, indicator 572 was actuatable between differentcolors or shades of colors to provide status information to operator 42.In some implementations, indicator 572 is actuatable between differentbrightness levels or is actuatable so as to flash or flash at differentfrequencies to provide status information to operator 42.

Indicators 574 comprise speakers/microphones. In the exampleillustrated, indicators 574 are located on underside of roof 620proximate steering console 618. Indicators 574 provide audible statusinformation to an operator remote from vehicle 524. In someimplementations in which indicators 574 also serve as microphones,indicators 574 may serve as input devices for the remote operator,whereby the operator may provide audible instructions or commands andwherein processor 32 uses speech recognition to identify such commandsand carry out such commands.

In some implementations, lights 526 may serve as additional indicators,wherein a color, brightness, blinking frequency, or the like of suchlights 526 may be controlled to provide status information to theoperator 42. In some implementations, additional visible indicators,such as light emitting diode lights, light bars or the like may beutilized to provide status information based upon the current state ofvehicle 524, its implements 225, its components 300, 302, 304, 306, 308,310, 312 and/or the positioning of operator 42 or the positioning ofinfluence 225 as based upon images captured by cameras 530.

State/feedback instructions 568 comprise software, code or logicelements on a circuit board provided in the non-transitorycomputer-readable medium 540. Instructions 568 direct processor 32 tooutput various control signals controlling the actuation or state ofindicators 570, 572 and 574. For example, processor 32, followinginstructions 568, may indicate a first state of vehicle 524 by providingindicator 572 with a first brightness, color, on/off state and/orblinking frequency and may indicate a second different state of vehicle524 by providing indicator 572 with a second different brightness,color, on/off state and/or blinking frequency. For example, indicator572 may be illuminated to have a green color when traveling forward andilluminated to have a red color when stopped. By way of another example,indicator 572 may be illuminated to have a green color when the powertakeoff is operating or when an implement is being powered and may havea red color when the power takeoff is no longer operating or when animplement is no longer being powered or driven.

Processor 32, following instructions 568, may indicate a first state ofvehicle 524 or second state of vehicle 524 by displaying graphics ortext on one or multiples of indicators 570. Such status informationprovided by indicators 570, 572, and 574 may include the speed or rateat which the vehicle is traveling, the speed or state of an implementand/or the state of any of lights 300, steering unit 302, propulsionunit 304, PTO unit 308, brakes 310 and/or auxiliary unit 312. Suchfeedback or status information provided by indicators 570, 572 and 574may include a confirmation of receipt or capture of gestures from theoperator (either operator anatomy 44 and/or input device 248), aconfirmation of recognition of such gestures, an indication that suchcommands are about to be executed, a request for the operator to repeatsuch gestures or to move so as to be more centrally located within thefield of view of cameras 530 when providing such gestures, or anindication that the commands associated with such gestures will not becarried out given the current state of vehicle 524 or the operator'sposition relative to vehicle 524 and/or its implements. Differentindicators may be utilized to provide different types of statusinformation to the operator.

In one implementation, processor 32, following instructions contained inmedium 540, utilizes images from camera 530-1 to identify thepositioning of rows of plants and to output control signals to steeringunit 302 and propulsion unit 304 to automatically drive vehicle 524 (andany attachment/implement 225) between and along the rows of plants (suchas crop plants, trees and the like). In one implementation, processor32, following instructions contained in medium 540, utilizes images fromcamera 530-1 to identify the positioning or location of operator 42 andthe movement of operator 42. Processor 32, following the instructionscontained in medium 540, may further block or allow other commands fromoperator 42 (based upon input gestures) based upon the position ormovement of operator 42. In some implementations, processor 32,following instructions contained in medium 540, may output controlsignals causing propulsion unit 304 and steering unit 302 to movevehicle 524 so as to follow the movement of operator 42 at a preselectedor operator selected distance. In some implementations, processor 32 maycontrol propulsion unit 304 and brakes 310 to substantially match thespeed at which the operator is moving. In some implementations,processor 32, following instructions contained in medium 540, mayutilize images captured by any of cameras 530 to identify animals orother obstructions, wherein processor 32 outputs control signals tosteering unit 302 and propulsion unit 304 to control the movement ofvehicle 524 so as to avoid such animals or obstructions. In someimplementations, processor 32 may utilizes signals from any of cameras530 to control the lighting provided by lights 624, 626. In someimplementations, processor 32 may utilize the signals from any ofcameras 530 and additional signals from a provided global positioningsystem to automatically, without operator intervention, drive vehicle524 to and from a worksite or field, to or from a storage lot, shed,garage or the like (a home location) for vehicle 524 or to or from acharging site or location for charging battery 604.

In some implementations, processor 32 may utilize the identifiedpositioning of operator 42 or of animals or other obstructions so as tocontrol brakes 310, PTO unit 306, auxiliary unit 312 orattachment/implement 225. For example, in one circumstance, theattachment/implement 225 may comprise a sprayer spraying herbicides,insecticides, fungicides or the like. In response to the detection ofthe presence of an operator or animal, processor 32 may temporarilycease the movement of vehicle 524 and/or the spraying operation untilthe operator animal is a predefined distance from the vehicle 524 or itsimplement 225. In some implementations, processor 32 may automaticallycease the operation of power take off 623 in response to images fromcameras 530 indicating that the operator, another person, or an animalare within a predefined distance from the power take off 623. In someimplementations, processor 32, following instructions contained inmedium 540, may utilize images captured by any of cameras 530 (plus anyother sensors divided on vehicle 524) to control the actuation of anattachment/implement 225. For example, processor 32 may identify thevarious locations of feed troughs and may control the actuation of anauger or other device of a pulled or attached implement 225 to unloadfeed at particular times and locations into the feed troughs. As aresult, processor 32 facilitates the automation of tasks.

In some implementations, indicators 570 or 572 may provide informationto an operator 42 in circumstances where the operator's currentidentified position would prevent him or her from viewing or determiningsuch information. For example, an operator positioned at the front ofvehicle 524 may be fired with information on indicators 570-1 or 570-2about the state of an implement 225 at the rear of vehicle 524. Anoperator positioned at the rear of vehicle 524 or at one side of vehicle524 may be provided with status information on selected indicators 570about the state of an implement, another operator or environment at thefront of vehicle 524 or at the other side of vehicle 524. As a result,system 520 provides an operator remote from vehicle 524 with informationthat may not otherwise be viewable given the operator's current positionrelative to vehicle 524.

Although the present disclosure has been described with reference toexample implementations, workers skilled in the art will recognize thatchanges may be made in form and detail without departing fromdisclosure. For example, although different example implementations mayhave been described as including features providing various benefits, itis contemplated that the described features may be interchanged with oneanother or alternatively be combined with one another in the describedexample implementations or in other alternative implementations. Becausethe technology of the present disclosure is relatively complex, not allchanges in the technology are foreseeable. The present disclosuredescribed with reference to the example implementations and set forth inthe following claims is manifestly intended to be as broad as possible.For example, unless specifically otherwise noted, the claims reciting asingle particular element also encompass a plurality of such particularelements. The terms “first”, “second”, “third” and so on in the claimsmerely distinguish different elements and, unless otherwise stated, arenot to be specifically associated with a particular order or particularnumbering of elements in the disclosure.

1-24. (canceled)
 25. A vehicle control system comprising: a vehicle; asensor carried by the vehicle; a processor; and a non-transitorycomputer-readable medium to direct the processor, the instructions theinstructions comprising: remote operator input sensing instructions toobtain a sensed input sensed by the sensor from an operator remote froma vehicle; input recognition instructions to recognize and associate thesensed input with a vehicle action; and input response controlinstructions to output control signals to the vehicle to cause thevehicle to carry out the vehicle action.
 26. The system of claim 25,wherein the sensed input comprises movement and/or positioning of ananatomy of the operator.
 27. The system of claim 26, wherein theinstructions further comprise operator identification and authorizationinstructions to verify authorization of the operator to provide inputfor the vehicle.
 28. The system of claim 28 further comprising a storedauthorized operator image, wherein the operator identification andauthorization instructions are to direct the processor to compare acaptured image of the operator to the stored authorized operator image.29. The system of claim 28 further comprising a stored authorizationgesture value, wherein the operator identification and authorizationinstructions are to direct the processor to compare a sensed gesturefrom the operator to the stored authorization gesture value.
 30. Thesystem of claim 28 further comprising a microphone and a stored auditoryauthorization value, wherein the operator identification andauthorization instructions are to direct the processor to compare anauditory input from the operator received through the microphone to thestored auditory authorization value.
 31. The system of claim 25 furthercomprising an input device, wherein the sensed input comprises movementand/or positioning of the input device by the operator.
 32. The systemof claim 31, wherein the instructions further comprise operatoridentification and authorization instructions to verify authorization ofthe operator to provide input for the vehicle.
 33. The system of claim32 further comprising a stored authorized operator image, wherein theoperator identification and authorization instructions are to direct theprocessor to compare a captured image of the operator to the storedauthorized operator image.
 34. The system of claim 32 further comprisinga stored authorization gesture value, wherein the operatoridentification and authorization instructions are to direct theprocessor to compare a sensed gesture from the operator to the storedauthorization gesture value.
 35. The system of claim 32 furthercomprising a stored auditory authorization value, wherein the operatoridentification and authorization instructions are to direct theprocessor to compare an auditory input from the operator to the storedauditory authorization value.
 36. The system of claim 31, wherein theinstructions further comprise input device identification andauthorization instructions to verify authorization of the input deviceto provide input for the vehicle.
 37. The system of claim 31, whereinthe input device comprises a display to present, to the remote operator,images captured by a camera carried by the vehicle.
 38. The system ofclaim 37, wherein the input device comprises a first face comprising aninput indicia recognizable by the processor following the inputrecognition instructions and a second opposite face comprising thedisplay.
 39. The system of claim 31, wherein input device comprises amanually held and positionable input device.
 40. The system of claim 31,wherein input device comprises a smart phone.
 41. The system of claim25, wherein the operator remote from the vehicle is standing on groundproximate the vehicle.
 42. The system of claim 25, wherein the vehiclecomprises a tractor.
 43. The system of claim 25, wherein the vehicleaction is selected from a group of vehicle actions consisting of:forward velocity, backward velocity, left/right direction, braking,lights (nightlights, running lights, spotlights), signal, sounds (horn,loudspeaker), warning (flashing lights, hazard lights), implementspecific actions (left sprayer on/off, right sprayer on/off), power takeup, moving a discharge spout; turning on/off a power take off; adjustinga speed of a power take off; and raising/lowering of an attachment tothe vehicle.
 44. The system of claim 25, wherein the vehicle comprises aharvester and when the vehicle action comprises repositioning of adischarge spout of the harvester.
 45. The system of claim 25, whereinthe vehicle comprises a tractor and when the vehicle action comprisesadjustment of a power take off of the tractor.
 46. The system of claim52, wherein the vehicle comprises a tractor having a front/rearattachment and wherein the vehicle action comprises adjustment of apositioning of the front/rear attachment.
 47. The system of claim 25,wherein the instructions further comprise operator positionidentification instructions to direct the processor to identify apositioning of the remote operator relative to the vehicle and to outputa notification to the operator recommending that the operator moverelative to the vehicle based upon the positioning.
 48. The system ofclaim 25, wherein the instructions further comprise operator positionidentification instructions to direct the processor to identifypositioning of the remote operator relative to the vehicle and toautomatically interrupt the vehicle action corresponding to the sensedoperator input based upon the positioning and the vehicle action beingrequested.
 49. The system of claim 25, wherein the instructions and theprocessor are part of at least one neural network.
 50. The system ofclaim 25 further comprising a stored database corresponding differentsensed inputs to different vehicle actions.
 51. The system of claim 25further comprising a feedback system carried by the vehicle to providedifferent states of illumination to indicate different states of thevehicle to an operator remote from the vehicle.
 52. The system of claim25, wherein the sensor comprises a camera. 53-76. (canceled)